{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* By: Illya Barziy\n",
    "* Email: illyabarziy@gmail.com\n",
    "* Reference: __Backtesting__ _by_ Campbell R. Harvey _and_ Yan Liu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Haircut Sharpe Ratios and Profit Hurdle algorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This description is based on the paper by _Campbell R. Harvey_ and _Yan Liu_ __“Backtesting”__  [available here](https://papers.ssrn.com/abstract_id=2345489).\n",
    "\n",
    "More descriptions of practical applications of the method are available in the article by _Campbell R. Harvey, Yan Liu_ and _Howard Moore_ __“Practical Applications of Backtesting”__  [available here](https://pa.pm-research.com/content/3/4/1.2)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A General overview of the framework"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is a common practice to discount reported Sharpe ratios by 50% as a result of data mining. The authors of the research developed an analytical way to determine the haircut's magnitude. \n",
    "\n",
    "The haircut is the percentage difference between the original Sharpe ratio and the Sharpe ratio adjusted to the effect of data mining.\n",
    "\n",
    "The authors explain that their framework relies on the concept of multiple testing. \n",
    "- If a a set of data $X$ explains $Y$ and the relation is significant with t-ratio of 2.0 (it has a probability value of 0.05), we refer to it as a single test. \n",
    "- If multiple sets of data $X_1, X_2, .., X_n$ explain $Y$, the same criteria for significance cannot be used (Some of the variables can produce t-ratios 2.0 and higher). Then, what is the appropriate cut off for statistical significance? \n",
    "\n",
    "Generally speaking, with a higher number of sets, the t-ratio is also higher."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When a strategy produces a Sharpe ratio, it's transformed into a t-ratio and then to p-value that takes into account multiple testing. \n",
    "\n",
    "In order to use the framework, one has to decide on the number of previous tests. In the research of _Harvey, C.R._, _Y. Liu_, and _H. Zhu_ __“… and the Cross-section of\n",
    "Expected Returns.”__  [available here](https://faculty.fuqua.duke.edu/~charvey/Research/Published_Papers/P118_and_the_cross.PDF) it was documented that at least 316 factors explaining the cross-sectional patterns in equity returns were found.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the provided approach, the haircut is nonlinear and the marginal Sharpe ratios are heavily penalized in comparison to high Sharpe ratios. Researchers state that it has economic sense, as strategies with high Sharpe ratios have a higher probability of being true discoveries.\n",
    "\n",
    "Researchers point to the following caveatas of the method:\n",
    "- High Sharpe ratios may be a result of non-normal distribution of returns. Therefore, Sharpe ratios should be viewed in the context fo the distribution of returns.\n",
    "- Sharpe ratios are not the only measures of risk, hovever the approach also applies to information ratios.\n",
    "- Need for determining the significance level for multiple testing.\n",
    "- Need to choose between the adjustment methods used in the framework provided (there are four of them - three separate ones and an average of them).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method description"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This part is a direct quote from the paper __“Backtesting”__ with minor comments.\n",
    "\n",
    "Let $r_t$ denote the return for an investment strategy between time $t-1$ and $t$. The strategy can consist of returns from both long and short positions. \n",
    "\n",
    "In order to conclude if the strategy is able to maintain true profits, a statistical test is formed to see if the expected excess returns are different from zero. \n",
    "\n",
    "Given a set of returns $(r_1, r_2, .., r_T)$, we denote $\\mu$ as the mean and $\\sigma$ as the standard deviation. T-statistic to test the null-hypothesis that the average return is zero is:\n",
    "\n",
    "$$t-statistic = \\frac{\\mu}{\\sigma/\\sqrt{T}}$$\n",
    "\n",
    "_The returns are assumed to be i.i.d. normal_, then the described t-statistic follows a t-distribution with $T-1$ degrees of freedom. This way we can assess the statistical significance of the investment strategy. \n",
    "\n",
    "At the same time, the Sharpe ratio is defined as:\n",
    "\n",
    "$$SR = \\frac{\\mu}{\\sigma}$$\n",
    "\n",
    "Therefore, based on the previous equation, \n",
    "\n",
    "$$SR = \\frac{t-ratio}{\\sqrt{T}}$$\n",
    "\n",
    "This shows that a higher Sharpe ratio implies higher t-statistic, which implies higher significance level (with fixed T).\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To adjust the Sharpe ratio for data mining bias, first we calculate the p-value of a single test:\n",
    "\n",
    "$$p^s = Pr(|r|>t-ratio) = Pr(|r|>SR*\\sqrt{T})$$, where $r$ is a random variable of a t-sistribution.\n",
    "\n",
    "This metric doesn't make sense when hundreds of strategies were tested only the most profitable is presented. \n",
    "\n",
    "If $N$ strategies were tested (and we assume the test statistics for $N$ strategies to be independent), under the null hypothesis that none of the strategies can generate non-zero returns, multiple testing p-value is:\n",
    "\n",
    "$$p^M = Pr(max\\{|r_i|, i = 1, .., N\\}>t-ratio) = 1 - \\prod^N_{i=1}Pr(|r_i|\\le t-ratio) = 1 - (1 - p^S)^N$$\n",
    "\n",
    "For $N=10, p^S=0.05$ whereas $p^M=0.401$. Multiple testing greatly reduces the statistical significance of a single test. \n",
    "\n",
    "Equating the p-value of a single test to $p^M$ will provide the equation for calculating the adjusted (haircut) Sharpe ratio $HSR$:\n",
    "\n",
    "$$p^M = Pr(|r|>HSR * \\sqrt{T})$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__A numerical example:__ \n",
    "\n",
    "$T = 200$ - ten years of monthly observations, $SR = 0.75$ - observed annual Sharpe ratio of $0.75$ and p-value of $0.0008$ in a single test.  When we assume the number of other strategies tested $N = 200$ and, therefore $p^M = 0.15$, we can calculate the adjusted Sharpe ratio $HSR = 0.32$, thus being reduced by $60\\%$ (haircut). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This calculation is true when N strategies are independent, however, this approach is not applicable for real-life cases. For this reason, in the paper _Harvey, C.R._, _Y. Liu_, and _H. Zhu_ __“… and the Cross-section of\n",
    "Expected Returns.”__  [available here](https://faculty.fuqua.duke.edu/~charvey/Research/Published_Papers/P118_and_the_cross.PDF) authors provide a multiple testing framework to find the appropriate p-value adjustment. This model is referred to as the HLZ model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The HLZ model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This part is a direct quote from the paper __“Backtesting”__ with minor comments.\n",
    "\n",
    "This model adjusts p-values for multiple testing taking into account that the strategies are not independent. It consists of three methods.\n",
    "\n",
    "The HLZ model is needed to estimate the underlying distribution for factor returns. It takes into account the correlation between the strategies based on 316 factors studied in the HLZ paper. As the input, the HLZ model uses only the average correlation between strategy returns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### __Bonferroni method__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, the p-values are ordered in ascending order.\n",
    "\n",
    "$$p_{(1)} \\le p_{(2)} \\le ... \\le p_{(M)}$$\n",
    "\n",
    "This method adjusts each p-value equally - inflates the original p-value by the number of tests $M$:\n",
    "\n",
    "$$p^{Bonferroni}_{(i)} = min[M*p_{(i)}, 1], i=1, .., M$$\n",
    "\n",
    "__A numerical example:__\n",
    "\n",
    "We observe $M = 6$ strategies with an ordered sequence of p-values of $(0.005, 0.009, 0.0128, 0.135, 0.045, 0.06)$. Under the single tests, five of the strategies are significant. Under the Bonferroni adjustment, however, p-values are \n",
    "\n",
    "$$p^{Bonferroni}_{(1)} = min[6p_{(1)}, 1] = 6p_{(1)} = 0.03$$\n",
    "\n",
    "$$p^{Bonferroni}_{(2)} = min[6p_{(2)}, 1] = 6p_{(2)} = 0.054$$\n",
    "\n",
    "$$p^{Bonferroni}_{(3)} = min[6p_{(3)}, 1] = 6p_{(3)} = 0.0768$$\n",
    "\n",
    "$$p^{Bonferroni}_{(4)} = min[6p_{(4)}, 1] = 6p_{(4)} = 0.081$$\n",
    "\n",
    "$$p^{Bonferroni}_{(5)} = min[6p_{(5)}, 1] = 6p_{(5)} = 0.27$$\n",
    "\n",
    "$$p^{Bonferroni}_{(6)} = min[6p_{(6)}, 1] = 6p_{(6)} = 0.36$$\n",
    "\n",
    "Which makes only the first strategy significant under Bonferroni method adjustments. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### __Holm method__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the same ordered set of p-values:\n",
    "\n",
    "$$p_{(1)} \\le p_{(2)} \\le ... \\le p_{(M)}$$\n",
    "\n",
    "The Holm method with $M$ tests adjusts p-values as follows:\n",
    "\n",
    "$$p^{Holm}_{(i)} = min[max_{j \\le i} \\{(M-j+1)*p_{(j)}, 1], i = 1, .., M$$\n",
    "\n",
    "The Holm method starts with the smallest p-value.\n",
    "\n",
    "__A numerical example:__\n",
    "\n",
    "Using the set of p-values from the previous example ($M = 6$,  $(0.005, 0.009, 0.0128, 0.135, 0.045, 0.06)$):\n",
    "\n",
    "$$p^{Holm}_{(1)} = 6 * p_{(1)} = 0.03$$\n",
    "\n",
    "$$p^{Holm}_{(2)} = max[6p_{(1)}, 5p_{(2)}] = 5p_{(2)} = 0.045$$\n",
    "\n",
    "$$p^{Holm}_{(3)} = max[6p_{(1)}, 5p_{(2)}, 4p_{(3)}] = 4p_{(3)} = 0.0512$$\n",
    "\n",
    "$$p^{Holm}_{(4)} = max[6p_{(1)}, 5p_{(2)}, 4p_{(3)}, 3p_{(4)}] = 4p_{(3)} = 0.0512$$\n",
    "\n",
    "$$p^{Holm}_{(5)} = max[6p_{(1)}, 5p_{(2)}, 4p_{(3)}, 3p_{(4)}, 2p_{(5)}] = 2p_{(5)} = 0.09$$\n",
    "\n",
    "$$p^{Holm}_{(6)} = max[6p_{(1)}, 5p_{(2)}, 4p_{(3)}, 3p_{(4)}, 2p_{(5)}, p_{(6)}] = 2p_{(5)} = 0.09$$\n",
    "\n",
    "Which makes the first two strategies significant under the Holm method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### __Benjamini, Hochberg and Yekutieli (BHY) method__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the same ordered set of p-values:\n",
    "\n",
    "$$p_{(1)} \\le p_{(2)} \\le ... \\le p_{(M)}$$\n",
    "\n",
    "Starting from the highest p-value, the adjusted values are calculated as:\n",
    "\n",
    "$$ p^{BHY}_{(i)} =\n",
    "  \\begin{cases}\n",
    "    p_{(M)}       & \\quad \\text{if } i=M \\text{,}\\\\\n",
    "    min[p^{BHY}_{(i)}, \\frac{M*c(M)}{i}p_{(i)}]  & \\quad \\text{if } i \\le M-1 \\text{,}\n",
    "  \\end{cases}$$\n",
    "  \n",
    "where $c(M) = \\sum^{M}_{j=1}\\frac{1}{j}$.\n",
    "\n",
    "Starting from the highest p-value, the method moves towards the smallest through pairwise comparison. \n",
    "\n",
    "__A numerical example:__\n",
    "\n",
    "Using the set of p-values from the previous example ($M = 6$,  $(0.005, 0.009, 0.0128, 0.135, 0.045, 0.06)$):\n",
    "\n",
    "$$c(M) = \\sum^{6}_{j=1}\\frac{1}{j} = 2.45$$\n",
    "\n",
    "$p^{BHY}_{(6)} = p_{(6)} = 0.06$$\n",
    "\n",
    "$$p^{BHY}_{(5)} = min[p^{BHY}_{(6)}, \\frac{6*2.45}{5}p_{(5)}] = p^{BHY}_{(6)} = 0.06$$\n",
    "\n",
    "$$p^{BHY}_{(4)} = min[p^{BHY}_{(5)}, \\frac{6*2.45}{4}p_{(4)}] = \\frac{6*2.45}{4}p_{(4)} = 0.0496$$\n",
    "\n",
    "$$p^{BHY}_{(3)} = min[p^{BHY}_{(4)}, \\frac{6*2.45}{3}p_{(3)}] = p^{BHY}_{(4)} = 0.0496$$\n",
    "\n",
    "$$p^{BHY}_{(2)} = min[p^{BHY}_{(3)}, \\frac{6*2.45}{2}p_{(2)}] = p^{BHY}_{(3)} = 0.0496$$\n",
    "\n",
    "$$p^{BHY}_{(1)} = min[p^{BHY}_{(2)}, \\frac{6*2.45}{1}p_{(1)}] = p^{BHY}_{(2)} = 0.0496$$\n",
    "\n",
    "Which makes the first four strategies significant under the BHY method.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### __Comparison of the methods__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Comparing the Bonferroni and Holm methods, $p^{Holm}_{(i)} \\le p^{Bonferrroni}_{(i)}$ for any $i$.\n",
    "\n",
    "Making Bonferroni method a tougher one comparing to Holm method. \n",
    "\n",
    "The BHY method is similar to the Bonferroni and Holm methods when the $N$ - number of multiple tests is small (around $N=10$). At this $N$ BHY penalizes strategies with higher Sharpe ratios a bit harder than Bonferroni and Holm.\n",
    "\n",
    "However, at $N=100$ BHY penalizes strategies at a significantly smaller rate compared to Bonferroni and Holm methods. \n",
    "\n",
    "Graphs with haircuts of methods depending on $N$ are availavle in the paper of _Campbell R. Harvey_ and _Yan Liu_, the link is attached at the beginning of the notebook.\n",
    "\n",
    "__Due to its nature, authors suggest using the BHY method in financial applications.__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Results of the algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The paper provides a comparison of Bonferroni Sharpe ratio adjustments for three types of strategies:\n",
    "- Earnings-to-price ratio (E/P)\n",
    "- Momentum (MOM)\n",
    "- Betting against beta factor (BAB)\n",
    "\n",
    "The above strategies had significant differences in annual Sharpe ratios. From the smallest of E/P ($SR = 0.43$) to MOM ($SR = 0.67$) and to the biggest BAB ($SR = 0.78$).\n",
    "\n",
    "With the increase of the multiple tests number the haircut of Sharpe ratio has increased. For example, at $N = 10$, haircut from the Bonferroni method is $26.6\\%$ for E/P, whereas for $N = 100$ it's already $61.6\\%$.\n",
    "\n",
    "It can also be seen that higher Sharpe ratios are penalized at a much lower rate. At $N = 100$ E/P haircut is $61.6\\%$ and the BAB haircut is only $9.3\\%$.\n",
    "\n",
    "The table with a full comparison of results is available in the paper of _Campbell R. Harvey_ and _Yan Liu_, the link is attached at the beginning of the notebook.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Haircut Sharpe Ratios algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. We are given the observed Sharpe ratio $SR$ in $T$ periods, based on this information we can calculate the p-value of a single test $p^S$. \n",
    "\n",
    "2. Assuming that $N$ other strategies have been tried and that the average correlation of returns from the strategies is $\\rho$ , we use the HLZ model to generate $N$ number of t-statistics from the model. We also transform the calculated $p^S$ to a t-statistic.\n",
    "\n",
    "3. This $N+1$ t-statistics are transformed again to p-values, taking into account the data mining adjustment.\n",
    "\n",
    "4. This set of $N+1$ p-values are fed to two models described above (Holm and BHY) to get the adjusted p-values with each of the methods. (Bonferroni adjustment is calculated using only the $p^S$ and $N$)\n",
    "\n",
    "5. The steps 2-4 are repeated multiple times (simulations). \n",
    "\n",
    "6. For each of the two methods, we eventually have a set of $p^M$ values adjusted. The median of this set is the final adjusted p-value of the method. So, we obtained p-values for every of the three methods. We then calculate the average p-value as the Average of the methods.\n",
    "\n",
    "7. The obtained p-values of each method can be then transformed back to Sharpe ratios and the haircuts can be calculated (Using the equation that shows the relation of $p^M$ with $HSR$, provided at the beginning of the notebook)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Profit Hurdle algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way to pose the problem. \n",
    "\n",
    "1. We are given the significance level $p$, strategy volatility $\\sigma$, the number of observations $T$, and number of tests that have been concluded $T$.\n",
    "\n",
    "2. Using the HLZ model, we generate $N$ t-statistics assuming that the average correlation of returns is $\\rho$.\n",
    "\n",
    "3. Using two methods (Holm and BHY) we calculate the threshold t-statistic that matches the $p$ significance level.\n",
    "\n",
    "4. The steps 2-3 are repeated multiple times (simulations).\n",
    "\n",
    "5. For the two methods (Holm and BHY) we have a set of t-statistics. We then take the median of t-statistics in each set and call it a t-statistic for the method. T-ststistic for Bonferroni is calculated based on $p$ and $N$, as in the previous algorithm (Haircut Sharpe Ratios). \n",
    "\n",
    "6. The obtained t-statistics of each method can be then transformed to mean monthly returns. (Using the equation that shows the relation of t-statistic with mean of the returns, provided at the beginning). We then calculate the average mean monthly return as the Average of the methods returns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usage of the algorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Haircut Sharpe Ratios"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Using the example from the paper.__\n",
    "\n",
    "We have a set of monthly returns from a strategy for 10 years. This makes $T=120$ total observations. The annualized Sharpe ratio observed from this returns is $SR = 1$. The Sharpe ratio is not adjusted to the autocorrelation of returns, but from the data, we've calculated that the autocorrelation coefficient (in the same frequency as the returns observations - monthly) is $\\rho_a = 0.1$. We have concluded 100 other tests (tested 100 other strategies). The average correlation of returns across strategies is $\\rho = 0.4$.\n",
    "\n",
    "We want to determine, what will be the appropriate haircuts to take data mining into account using all three methods?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mlfinlab as ml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Note:__ this algorithm requires some time to run the simulations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The adjusted Sharpe ratio using the Bonferroni method is 0.23173088626909502\n",
      "The adjusted Sharpe ratio using the Holm method is 0.26184765766084483\n",
      "The adjusted Sharpe ratio using the BHY method is 0.4324231267610195\n",
      "The average adjusted Sharpe ratio of the methods is 0.2970209752831795\n"
     ]
    }
   ],
   "source": [
    "# Creating an object and specifying the desired level of simulations to do\n",
    "# for Haircut Sharpe Ratios and Profit Hurdle in the Holm and BHY methods.\n",
    "backtesting = ml.backtests.CampbellBacktesting(simulations=2000)\n",
    "\n",
    "# Calculating the adjusted Sharpe ratios and the haircuts.\n",
    "haircuts = backtesting.haircut_sharpe_ratios(sampling_frequency='M', num_obs=120, sharpe_ratio=1,\n",
    "                                             annualized=True, autocorr_adjusted=False, rho_a=0.1,\n",
    "                                             num_mult_test=100, rho=0.4)\n",
    "\n",
    "# Adjusted Sharpe ratios by the method used.\n",
    "print('The adjusted Sharpe ratio using the Bonferroni method is', haircuts[1][0])\n",
    "print('The adjusted Sharpe ratio using the Holm method is', haircuts[1][1])\n",
    "print('The adjusted Sharpe ratio using the BHY method is', haircuts[1][2])\n",
    "print('The average adjusted Sharpe ratio of the methods is', haircuts[1][3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also output the haircuts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Sharpe ratio haircut using the Bonferroni method is 74.59774849897369\n",
      "The Sharpe ratio haircut using the Holm method is 71.29635948860347\n",
      "The Sharpe ratio haircut using the BHY method is 52.59794152736317\n",
      "The average Sharpe ratio haircut of the methods is 67.44067380615853\n"
     ]
    }
   ],
   "source": [
    "# Sharpe ratio haircuts.\n",
    "print('The Sharpe ratio haircut using the Bonferroni method is', haircuts[2][0])\n",
    "print('The Sharpe ratio haircut using the Holm method is', haircuts[2][1])\n",
    "print('The Sharpe ratio haircut using the BHY method is', haircuts[2][2])\n",
    "print('The average Sharpe ratio haircut of the methods is', haircuts[2][3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the BHY method has the smallest haircut comparing to other methods. This is exactly the expected behaviour when dealing with a big number of tests ($N=100$ in our case.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results match the results presented by the authors of the paper."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Profit Hurdle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Using the example from the paper.__\n",
    "\n",
    "We want to determine the Minimum Average Monthly Return that a proposed strategy must exceed at a significance level of $\\alpha = 0.05$. We assume the strategy to cover 20 years ($T = 240$ monthly observations) and have annual volatility of 10%. We also allow $N = 300$ other tests (strategies tested) with returns correlation of $\\rho = 0.4$.\n",
    "\n",
    "What is the Minimum Average Monthly Return for each of the methods described above?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Required Minimum Average Monthly Returns using the Bonferroni method is 0.3652177252402424\n",
      "Required Minimum Average Monthly Returns using the Holm method is 0.701533466971392\n",
      "Required Minimum Average Monthly Returns using the BHY method is 0.6868110733736568\n",
      "Required Minimum Average Monthly Returns using the average of the methods is 0.6212594549266116\n"
     ]
    }
   ],
   "source": [
    "# Calculating the Minimum Average Monthly Returns.\n",
    "monthly_ret = backtesting.profit_hurdle(num_mult_test=300, num_obs=240, alpha_sig=0.05,\n",
    "                                        vol_anu=0.1, rho=0.4)\n",
    "\n",
    "# Minimum Average Monthly Returns by the method used.\n",
    "print('Required Minimum Average Monthly Returns using the Bonferroni method is', monthly_ret[0])\n",
    "print('Required Minimum Average Monthly Returns using the Holm method is', monthly_ret[1])\n",
    "print('Required Minimum Average Monthly Returns using the BHY method is', monthly_ret[2])\n",
    "print('Required Minimum Average Monthly Returns using the average of the methods is', monthly_ret[3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the results, the lowest Minimum Average Monthly returns are given by the Bonferroni method at $0.365\\%$ monthly returns and the highest is from the Holm method at $0.7\\%$ monthly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook describes the Haircut Sharpe Ratios and Profit Hurdle algorithms and how they may be used in real-life applications.  \n",
    "\n",
    "The algorithms were originally presented by the authors _Campbell R. Harvey_ and _Yan Liu_ in the paper __“Backtesting”__  [available here](https://papers.ssrn.com/abstract_id=2345489).\n",
    "\n",
    "Key takeaways from the notebook:\n",
    "- Sharpe ratios should be adjusted to data mining and these algorithms provide an analytical way to determine the adjustment magnitude. \n",
    "- These algorithms take into account that the strategies in multiple testing are not independent.\n",
    "- Three methods are available to determine the adjustment. For the financial applications, authors suggest using the BHY method in financial applications.\n",
    "- The haircut is nonlinear and the marginal Sharpe ratios are heavily penalized in comparison to high Sharpe ratios. (In the provided example $61.6\\%$ haircut for $0.78$ Sharpe ratio and only $9.3\\%$ for $0.43$ Sharpe ratio.)\n",
    "- When the number of multiple tests is high, BHY penalizes strategies at a significantly smaller rate compared to Bonferroni and Holm methods. \n",
    "- To use the Haircut Sharpe Ratio algorithm, the following information is required:\n",
    "  - The sampling frequency of the returns\n",
    "  - Number of returns in a sample\n",
    "  - Sharpe ratio observed (annualized or not)\n",
    "  - Autocorrelation coefficient of returns (if Sharpe ratio was not adjusted for the autocorrelation)\n",
    "  - Number of other strategies tested\n",
    "  - Average correlation coefficient of the returns from other strategies\n",
    "- To use the Profit Hurdle algorithm, the following information is required:\n",
    "  - Number of other strategies assumed\n",
    "  - Number of returns in a sample\n",
    "  - Significance level\n",
    "  - Annual volatility of returns\n",
    "  - Average correlation coefficient of the returns from other strategies"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mlfinlab",
   "language": "python",
   "name": "mlfinlab"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
